Exploring Reality-Based Interaction through Whole-Body Movement

In this paper, we present a research agenda that focuses on employing whole-body movement as an interaction modality for accessing and manipulating abstract information. This investigation aims to answer various open questions including: What movement qualities could be computationally sensed and how could these qualities be mapped to control digital information? What computational models are required for interpreting simultaneous movements of multiple users? And, is there a universal gesture vocabulary for certain tasks? We briefly discuss three projects that combine whole-body movement with other themes of reality-based interaction such as naïve physics and social awareness in an attempt to answer these questions.